Visuals
Introduction Physical / Hardware On the AUV, there is two cameras, front and bottom. Coding The language used will be C++. Besides C++, you must also have a knowledge for coding in terms of Robot Operation System (ROS) and Open Source Computer Vision(OpenCV). OpenCV is a library for visual related code. Conclusion, Knowledge in * C++ * ROS * OpenCV We act primarily as a sensor to the robot. We publish (give) data to the navigation team, where they will subscribe (get) data and process the data according to the situation. Getting started So to code, you need * Odroid * Keyboard * Power Supply * Monitor Connect the wireless keyboard and cameras before powering up the odroid. After you power odroid, you will be led to Ubuntu desktop. To run any code, go to LXTermial and type 'roslaunch colortest.launch' *Tips* you can press tab twice to see possible suggestions Press ctrl + c to kill the program or ctrl + z. Sometimes alt+ tab is the only way to kill it. So when you type 'roslaunch colortest.launch' you are actually running a launch file, and the launch file runs the cpp code you have linked it to. To edit the launch file, you go to your desktop and click "File manager" and click open "Ivan.launch" for example. Coding The .cpp file directory for image processing is found in / home / odroid / catkin_ws / src / beginner_tutorials / src We will be covering on 3 mains codes * sp_auv_colortest.cpp * cal2017.cpp * Ivan.cpp sp_auv_colortest.cpp Summary *This code requires only 1 camera to be plugged in to work Colortest is the most important code, or the base code What this code does is find the H S V values of particular object you place under the camera. We are using H S V over R G B because H-S-V is less sensitive to light. Every color have its unique H-S-V values. You will input the H-S-V values into the code Ivan.cpp manually. After running the code, a image of this will pop out. Now, we only need to focus on 2 windows, which is trackbar and '' cambottom '' . There will be values running in the terminal. This shows the estimated HSV values inside the yellow box. The circle will move according to the center of object. Place any object under the camera and adjust the trackbar until the desired colour is white and everything is black. What you are doing is thresholding the values of the color. Lets say for instance, (digits) This means that anything within the range of this values, it will be binary '1'. (White in colour). Whereas anything outside the range will be binary '0' . (Black) Coding explaination cvCreateTrackbar("LowH", "Control", &iLowH, 179); This creats a trackbar for the trackbar window. The previous year they capped the values at 179. For the year of 2017, we are required to detect blue. Blue is after the range of 180-255. So i changed the value and uncapped it. I suspect the reason of doing this is because the swimming pool tiles are blue, and it isn't necessary to detect blue throughout the past 4 years. This is only my assumption. cal2017.cpp 'Summary' This code is made to auto assign ports numbers to the camera. What odroid does is it randomly assign port number every time it boots up. This results in the code thinking that the physically front camera is the bottom camera. Previously, before I made this code, they 'fixed' the problem with having to plug in and out the usb cables manually until the cameras are in the correct port numbers. With code, we will cover the bottom camera with a black/white colored paper. And we will flash the front camera with an orange/red object. Explanation if(countB >30) { cam_angle.data="bottom"; camera_pub_angle.publish(cam_angle); } if(countF >30) { cam_angle.data="front"; camera_pub_angle.publish(cam_angle); } So if bottom camera detects, countB / countF will count up. As there is a chance of detecting noise, I made the count more than 30 . This requires an object to be constantly there for about 3 seconds. After countB is above 30, it will send out the string, "bottom" into cam_angle. In the statemachine.cpp code, it will read the cam_angle. If it receieves " bottom " , it will send me an int '0' in cam_calibration . If it receieves " front " , it will send me an int '1' in cam_calibration . In the main code, "Ivan.cpp" it will process the data from "cam_calibration" and switch the usb assignment if neccessry. Ivan.cpp This is the actual code that will be running during the competition for the year of 2017. For this particular year, we are required to pass through a gate, and drop a ball inside blue bucket. Then we are required to knock down a flare which is yellow in color. So for this year, we are concern primary 4 colors, red,green,blue, yellow. To navigate through the gate , we will be detecting the red colored pole (On the left side of the gate) using front camera. We will be able to calculate the x coordinates of the red pole. Since we set the dimension of the camera to be 240 x 320, the range of the x coordinates spans from 0 - 320. To ensure that we do not hit the gate while passing, we must maintain the x values at 80, with maybe a tolerance of +-10. After we pass through the gate, we will be switching over to the bottom camera, and we will be constantly searching for green. It is possible to directly search for the blue bucket, but the tiles of the swimming pool are blue, this may result in the camera mistaking the blue tiles as blue bucket and drop the ball immediately. In order to avoid that, we implemented a sequence must be established, first detecting the green mat before detecting the blue bucket. After dropping the ball, we will swim around to find the yellow flare. General coding line explantion inRange(framefront, Scalar(RLowH,RLowS,RLowV), Scalar(RHighH,RHighS,RHighV), imgThresholdedF); erode(imgThresholdedF, imgThresholdedF,getStructuringElement(MORPH_ELLIPSE, Size(5,5))); //Eliminate the small particles dilate(imgThresholdedF, imgThresholdedF,getStructuringElement(MORPH_ELLIPSE, Size(5,5))); erode(imgThresholdedF, imgThresholdedF,getStructuringElement(MORPH_ELLIPSE, Size(5,5))); dilate(imgThresholdedF, imgThresholdedF,getStructuringElement(MORPH_ELLIPSE, Size(5,5))); inRange - You need to input the upper and lower values of the HSV values so that it can threshold the values. Erode / dilate - used to remove small objects from the foreground ( noises in other words) VideoCapture cambottom(1); The integer '1' can be represent the number of camera there is. By sequence of 0,1,2,3... Assign 0 if there is only 1 camera. Assign 0 and 1 if there is 2 cameras.. so on if(ifgui) namedWindow("Thresholded Frame (Bottom)",1); ifgui was declared as true/false at the top. Recommend always set it to true. ( Previous year(2016) set it to false, resulting in no widow popping out. Took me a week to debug and realize you have to set it to 'true' to view camera. ) *no idea why called ifgui* oMoments = moments(imgThresholdedB); dM01 = oMoments.m01; dM10 = oMoments.m10; int posX = dM10/dArea; int posY = dM01/dArea; What it does is it counts all the x coordinates and y coordinates and add them all up. It then divides by the total to gain an average. This is used to find the average center of object. dArea = oMoments.m00; if(dArea >2000) These are used to calculate the area of the whites seen by the camera. If the area is more then 2000, it will drop the ball etc etc. According to my supervisor , the maximum should be 320 x 240 . This values are the dimension set at the beginning of the code . However, when I read the values from the terminal, it shows numbers in millions. My estimation is 20mil for everything white, so i used 5 million instead of 2k during the competition. cam_state.data="not detected"; camera_pub_state.publish(cam_state); cam_state.data is writing the string into cam_state. .publish is to send out the string, “ not detected” into camera_pub_state. Testing results Testing Testing the colour green Below is the set up, Green bucket, with a weight placed to keep the at the bottom. Used the side of the medium pool. Alternative way to test colours We are able to view and calibrate the HSV values of the colours using laptop. VNC downloaded on the laptop Putty downloaded on the laptop Github using odroid to download the VNC package. Connect the robot to the laptop using the underwater cable. When connected, launch putty and click on the word ‘’odroid’’ . It will launch a terminal and ask for the ID and password ID and password are both : odroid After you get in, enter 'x11vnc'. *Note its eleven, not L L . Press enter and it should start counting up from 1 to 5. After it stop counting, click on your VNC and launch it. Click odroid. It should bring you to odroid desktop. This VNC application creates a second monitor , so you can view the odroid screen from 5 meters away. (cable length) However, do take note that cable may be lose at times, and it may disconnect anytime. You can check the connection by right clicking the bottom left Wi-Fi signal and check if the Ethernet port is there. There should be 3 choices, Ethernet, Wi-Fi and Local Area Network. If there isn’t a choice of Ethernet restart the laptop. Do not plug in the Ethernet cable when booting up the laptop. It will cause the Ethernet driver to be missing. Previously what I heard was they brought the actual monitor and went there to calibrate. For the testing of the colors, let’s say there is the color ‘green’ but there can be light green, dark green, etc etc. So we are only to estimate the color of the object before competition. During the competition, you are given the actual object. Time is given to calibrate your camera and you are able find the values of the HSV of the actual color. Problems I encountered were I tried running the code on raspberry PI. I installed both Ubuntu and ROS on it. However, because the version of ROS doesn’t fit, the program could not run. The first attempt to do the auto port assignment was by reading values from a text file instead of using ros to communicate. I was only able to write into a text file but wasn’t able to no data from it . Difficult I faced were finding the ranges of the colour. Because colour differs from land than on surface, we must bring everything to the pool to test, to mimic the actual environment. The lighting of the weather also affects the reading, example, sunny day gives a different value compared with gloomy days.